Skip to content

Preserve the declared width of short raw and text fields - #60

Open
rupayon123 wants to merge 2 commits into
eerimoq:masterfrom
rupayon123:fix/pad-short-raw-text-fields-20261031
Open

rupayon123 wants to merge 2 commits into
eerimoq:masterfrom
rupayon123:fix/pad-short-raw-text-fields-20261031

Conversation

@rupayon123

@rupayon123 rupayon123 commented Oct 31, 2026 •

Copy link
Copy Markdown

Short raw and text fields in the Python implementation can produce fewer bits than their declared width. Round the missing byte count upward before the existing truncation and bit-order handling, so following fields keep their declared offset.

Following review, the native C backend now also zero-pads short or empty raw/text payloads at its supported byte-multiple widths. It uses the existing byte writers to preserve adjacent fields and untouched pack-into bits. The documented C restriction to byte-multiple field widths and existing type checks remain unchanged.

Native CPython 3.13.14 / macOS arm64 validation:

  • Full installed-wheel suite: 56 tests and 377 subtests pass; the Makefile unittest command passes all 56 tests.
  • The rebuilt cp313 wheel and loaded native extension identities are recorded. An independent C wire-format oracle passes all 12,320 observations across direct, compiled, dictionary and pack-into APIs, including unaligned offsets.
  • The Python implementation's original 1,488-observation field-width proof remains separate. No arbitrary-width C capability is added.

Existing raw bytearray ownership semantics are unchanged. Other hosted runtimes/platforms remain separate from these local checks.

@eerimoq

eerimoq commented Oct 31, 2026

Copy link
Copy Markdown
Owner

so C implementation still has the bug? please fix it as well in that case

@rupayon123

Copy link
Copy Markdown
Author

The C backend now zero-pads short and empty raw/text payloads too. It writes the available bytes, pads the remaining field bytes, and keeps the following field at the correct offset. The documented C restriction to byte-multiple raw/text field widths is unchanged.

I rebuilt and installed the CPython 3.13 wheel. The full suite passes (56 tests and 377 subtests), as does the unittest command. The new cases cover direct, compiled, dictionary and pack-into calls, including non-byte-aligned offsets and preservation of the surrounding destination bits. An independent wire-format check also passes all 12,320 observations on the installed C extension.

@eerimoq

eerimoq commented Nov 2, 2026

Copy link
Copy Markdown
Owner

remove new test file. make new tests easier to read an understand. less looping, fewer conditions. straight forward easy to read test code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants